New and innovative methods of interacting with computers are being developed to increase usability of computers. For example touch screen interfaces have been introduced that allow users to provide commands to a computer without a mouse and/or keyboard. However, some of these new methods are vulnerable to interpreting inadvertent user interactions as commands; such inadvertent interactions may be defined as any interaction detected by the touch screen interface without the user's consent. Inadvertent interactions may be caused by bumps, vibrations, accidental brushes by a user's hand or other physical objects, or the like, and may result in system malfunctions or operational errors.
An interface that is currently being developed, which provides greater immunity to inadvertent user interaction, is a gesture recognition interface. Gestures are generally hand movements. As referenced herein, a gesture has specific motion dynamics and is composed of one or more static or dynamic components of a user's palm and fingers. However, adoption of the gesture recognition interface also has constraints, such as requiring a high degree of preprocessing and complex algorithms to extract and analyze the user's interaction from the surrounding environment.
Gesture recognition interfaces depend upon three-dimensional (3D) imaging information sensed across an established visible sensing, volumetric, range. An extensive visible sensing range limits the accuracy, reliability, and precision of the system, and presents significant technology challenges for aircraft flight deck applications.
One proposed method of avoiding these issues relies on a user to hold or otherwise manipulate a controller or handheld device. These controllers may act as an extension of the body so that when a gesture is performed, the motions can be recognized by the system. However, in addition to requiring the user to hold a controller or other device in order to interact with a processor, only limited types of motions are recognized by the system.
To overcome some of the disadvantages of the aforementioned methods, a volumetric computing system has been proposed for use on, for example, an aircraft flight deck, to reduce inadvertent user interactions. A sensor is placed proximate a display device that is coupled to an interaction controller. This volumetric computing system enables users or developers to interact with the system to activate control functions without making physical contact with the system. An example of such a system is shown and described in U.S. patent application Ser. No. 13/777,737 filed Feb. 26, 2013 entitled “SYSTEM AND METHOD FOR INTERACTING WITH A TOUCH SCREEN INTERFACE UTILIZING A HOVER GESTURE CONTROLLER,” and assigned to the instant assignee, the teachings of which are hereby incorporated by reference. The system of the above patent application represents a significant improvement over the prior art.
Impediments to wide adoption of 3D gesture interfaces remain. Perhaps the most significant impediment to the wide adoption of 3D gesture interfaces resides in the need for the user to raise his or her arms and hands in order to input gestures into the system. This is tiresome, and diminishes the appeal of the interface. Another drawback is that the interacting hand is often in the user's line of sight. Users generally prefer to minimize any visual fixation on the interacting hand, and to be able to sit, with the arm of the interacting hand on arm-rest. Yet another drawback is that a system relying on one, extended, sensor volume to generate multiple valid 3D gesture interaction volumes for multiple users may be vulnerable to inadvertent or unintentional gesture input. The aforementioned preferences and vulnerabilities may hinder the practicality of the current gesture recognition systems in certain applications. Utilizing individual ergonomic and anthropometric measurements (for example, arm length, seat height, arm-rest height, and the like) to accommodate user preferences is desirable and would address some of the impediments to adoption of 3D gesture interfaces.
In view of the foregoing, an ergonomic three-dimensional, gesture based, multimodal interface for use in flight deck applications is desirable, this interface is hereinafter referred to as a “3D gesture interaction interface.” It would be desirable for the interface to operate when the pilot's arm is supported on an arm-rest, and function while the interacting hand is not in the line of sight of the pilot; these interactions are sometimes referred to as ‘arm-rest-support-gesture-interactions’. It would further be desirable for the interface construct a valid 3D gesture interaction volume (hereinafter referred to as “valid interaction volume”) either automatically (automatic mode) or by allowing the pilot to interact and customize according to ergonomic and anthropometric requirements and preferences (pilot-driven mode). In addition to reducing the probability of inadvertent activations of control functions, these improvements would reduce fatigue, improve productivity, and enhance the user's overall experience.